4 research outputs found
MaLP: Manipulation Localization Using a Proactive Scheme
Advancements in the generation quality of various Generative Models (GMs) has
made it necessary to not only perform binary manipulation detection but also
localize the modified pixels in an image. However, prior works termed as
passive for manipulation localization exhibit poor generalization performance
over unseen GMs and attribute modifications. To combat this issue, we propose a
proactive scheme for manipulation localization, termed MaLP. We encrypt the
real images by adding a learned template. If the image is manipulated by any
GM, this added protection from the template not only aids binary detection but
also helps in identifying the pixels modified by the GM. The template is
learned by leveraging local and global-level features estimated by a two-branch
architecture. We show that MaLP performs better than prior passive works. We
also show the generalizability of MaLP by testing on 22 different GMs,
providing a benchmark for future research on manipulation localization.
Finally, we show that MaLP can be used as a discriminator for improving the
generation quality of GMs. Our models/codes are available at
www.github.com/vishal3477/pro_loc.Comment: Published at Conference on Computer Vision and Pattern Recognition
202
PrObeD: Proactive Object Detection Wrapper
Previous research in object detection focuses on various tasks,
including detecting objects in generic and camouflaged images. These works are
regarded as passive works for object detection as they take the input image as
is. However, convergence to global minima is not guaranteed to be optimal in
neural networks; therefore, we argue that the trained weights in the object
detector are not optimal. To rectify this problem, we propose a wrapper based
on proactive schemes, PrObeD, which enhances the performance of these object
detectors by learning a signal. PrObeD consists of an encoder-decoder
architecture, where the encoder network generates an image-dependent signal
termed templates to encrypt the input images, and the decoder recovers this
template from the encrypted images. We propose that learning the optimum
template results in an object detector with an improved detection performance.
The template acts as a mask to the input images to highlight semantics useful
for the object detector. Finetuning the object detector with these encrypted
images enhances the detection performance for both generic and camouflaged. Our
experiments on MS-COCO, CAMO, CODK, and NCK datasets show improvement
over different detectors after applying PrObeD. Our models/codes are available
at https://github.com/vishal3477/Proactive-Object-Detection.Comment: Accepted at Neurips 202
Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Generated Images
State-of-the-art (SOTA) Generative Models (GMs) can synthesize
photo-realistic images that are hard for humans to distinguish from genuine
photos. We propose to perform reverse engineering of GMs to infer the model
hyperparameters from the images generated by these models. We define a novel
problem, "model parsing", as estimating GM network architectures and training
loss functions by examining their generated images -- a task seemingly
impossible for human beings. To tackle this problem, we propose a framework
with two components: a Fingerprint Estimation Network (FEN), which estimates a
GM fingerprint from a generated image by training with four constraints to
encourage the fingerprint to have desired properties, and a Parsing Network
(PN), which predicts network architecture and loss functions from the estimated
fingerprints. To evaluate our approach, we collect a fake image dataset with
K images generated by GMs. Extensive experiments show encouraging
results in parsing the hyperparameters of the unseen models. Finally, our
fingerprint estimation can be leveraged for deepfake detection and image
attribution, as we show by reporting SOTA results on both the recent Celeb-DF
and image attribution benchmarks